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ABSTRACT 

LISP70 is a descendant of LISP which emphasizes pattern-directed 
cotiiputation and extensibility. A function can be defined by a set of 
p.attei-n i-ei-iri'te rules as well as by the normal LAMBDA method.. New 
pfuiritG rules can be added to a previously defined function; thus 
l_ISP70 functions are said to be "extensible". It is possible to have 
the neu rules merged into the function automatically so that special 
cases are checked before general cases. Some of the facilities of 
the reur ite system are described and a variety of applications are 
demonstrated, 

NOTICE 

This is a limited circulation draft of a paper to be submitted 
to a conference or journal. No right to publicize its contents is 
qranted. 
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BACKGROUND 

During the past decade, LISP EIG] has been a principal 
programming language for artificial intelligence and other frontier 
applications of computers. Like other widely used languages, it has 
spanned many variants, each attempting to make one or more 
improvements. Among the aspects that have received particular 
attention are notation El, 10, 14,21] , control 

structure [4,11,17,191, data base management [12,17,223, interactive 
editing and debugging [24], and execution efficiency. 

A need for a successor to LISP has been recognized [3], and 
several efforts in this direction are under way. The approach being 
taken with TENEX-LISP is to begin with an excellent debugging 
system [23] and to add on flexible control structure [2] and A I go I - 
like notation. The approach taken by LISP70 and by the LISP-related 
l".CL [2G] is to begin with an extensible kernel language which users 
can tailor and tune to their own needs. 

"Tailoring" a language means defining facilities which assist in 
the solution of particular kinds of problems which may have been 
unanticipated by the designers of the kernel. "Tuning" a language 
means specifying more efficient implementations for statements which 
are executed frequently in particular programs. 
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A language that can be used on only one computer is not of 
universal utility; the ability to transfer programs between computers 
incr eases its value. However, a language that is extensible both 
upi-iard and downward is difficult to transport if downward extensions 
mention machine-dependent features [7,81. LISP70 generates code 
for an "ideal LISP machine" called "ML" and only the translation from 
ML to object machine language is machine-dependent. Thus, downward 
extensions can be factored into a machine-independent and a machine- 
dependent part, and during program transfer, the machine-dependent 
receding (if any) is clearly isolated. 

Among the improvements LISP70 makes to LISP are backtrack 
control structure [13], streaming [15], pattern-directed computation, 
and extensible functions. 

The subjects to be covered in the present paper are pattern- 
directed computation and extensible functions. 
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PATTERN-DIRECTED COflPUTATION 

Many of the data tranformat ions performed in LISP applications 
are more easily described by pattern matching rules than by 
alfjoi-ithms [12,17,22,25], In addition, pattern matching rules are 
yp|;>iopr iate for the description of input-output conversion, parsing, 
and camp i ling [201. LISP70 places great emphasis on "pattern rewrite 
rules" [5,6,13,27] as an alternative and adjunct to algorithms as a 
means of defining functions. 

A brief explanation of rewrite rule syntax and semantics will be 
presented nith some examples to demonstrate the clarity of the 
no tat i on. 

Each rule is of the form DEC-)REC. The DEC (decomposer) is 
matcl-ied against the "input stream". If it matches, then the REC 
(recomposer) generates the "output stream". 

A literal in a pattern is represented by itself if it i s an 
identifier or number, or preceded by a quote (*) if it is a special 

character. 

RULES OF SQUARE = 
2-^4, 
5 -^ 25, 
12 -^ 144 ; 
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A private, variable of the rule is represented by an identifier 

prefixed by a colon (:); it may be bound to only one value during 

operation of the rule. 

RULES OF EQUAL = 
:X :X -> T, 
:X :Y -^ NIL ; 

A list is represented by a pair of parentheses surrounding the 

representations of its elements. A segment of zero or more elements 

is represented by an ellipsis symbol (...). 

RULES OF CAR = 

(:X ...) -^ :X ; 

RULES OF CDR = 

(;X ...) -> (...) ; 

RULES OF CONS = 

:X (...) -^ (:X ...) ; 

RULES OF ATOn = 

(:X ...) -> NIL, 
:X -* T ;■ 

RU[£S OF APPEND = 

(...) (...) -* (... ...) ; 

If a segment needs a name, it is represented by an identifier 

pi-o.f i xcd by a doub I e-co I on ( : ; ) . 

RULES OF ASSOC = 

:X (... (:X ::Y) ...) -^ (jX ::Y), 
:X (...) ^ NIL ; 

A function F can be called in a pattern, passing it a single 
ai-nurnnnt ARG, by the construct: ARG@F (there are also ways of- passing 
several arguments to a function). 
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RULES OF LENGTH = 

{ ) ^0, 

(sX ...) -^ (...) ©LENGTH oADDl ; 
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LIST STinjCTURE TRANSFORhATIONS 



The folloi-iing set of rules defines a function t10VE_BL0CK of 
three arguments: a block to lie moved, a location to which it should 
be iiiovod, and a representation of the current world. The function 
moves block :B from its current location in the world to location 
:T0, and the transformed representation of the world is returned. 
RULES OF nOVE BLOCK = 



:B :T0 (. 
^ (. 


. (sTO 
. (:T0 




:B .. 
:B .. 






:B :T0 (. 
-^ {. 




:B 


, , ) , 


.. (:T0 ... ) . 
.. (;T0 ... :B) . 




:B :T0 (. 
-> (. 


.. (:T0 
. . (:T0 


... 


:B) ! 


.. ( ) . 




:B :T0 (. 
-. (. 


.. (,.. 
.. (... 


:B 


. . ) . 


.. (:T0 :B)), 




:B :T0 (. 
^ (Bl 


..) 

_0CK :B 


NOT 


IN (. 


..)) ©ERROR ; 





In the first case, the block is already where it belongs, so the 
world does not change; in the second, the block is moved to the 
right; in the third, to the left; in the fourth, the location : TO 
does not exist yet and is created; in the last case, :B is not in the 
world and the ERROR routine is called. . 



Functions such as M0VE_BL0CK have been used in a simple planning 
program written by one of the authors. Imagine writing MOVE_BLOCK as 
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an algorithm; it would require the use of auxiliary functions or of a 
PROG uith state variables and loops. Bugs would be more likely in 
thie algorithm because its operation would not be so transparent. 
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REPLACENENT 

If F is any function, then the construct <F> occurring in a DEC 
pattern signifies "replacement". This means that F is invoked to 
translate a substream of the input stream, and that substream is 
replaced by its translation. The altered input stream can then 
continue to be matched by the pattern to the right of <F>. 

The folloning example is from the NLISP compiler, which calls 
itself recursively to translate the condition and arms of an IF- 

statciiient to LISP; 

RULES OF ML ISP = 

IF <nLISP>:X THEN <riLISP>:Y ELSE <I1LISP>!Z 
-^ (COND (:X :Y) (T :Z)), 

IF <MLISP>:X THEN <nLISP>:Y 

-^ (COND (:X :Y) (T NIL)), 

IF <nLISP>;X 

-^ (niSSING THEN) ©ERROR, 

IF -^ (ILLEGAL EXPRESSION AFTER IF) ©ERROR ; 

Here is another example. The predicate PALINDROME is true iff 
the input stream is a mirror image of itself, i.e., if the left and 
right ends arc equal and the middle is itself a palindrome. 
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RULES OF PALINDROriE = :X ^ T, 

:X :X -* T, 

;X <PALINDROnE>T sX -^ T, 

■^ NIL ; 

The replacement facility provides both the non-terminal symbols 
of syntactic parsing and the "actors" of PLANNER [12]. 
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EXTENSIBLE FUNCTIONS 

Ngm i-uIog may be added to an existing set of rewrite rules under 

pragraiii control; thus, any compiler table or any other system of 

roiirite. rules can be extended by the user. For this reason, a set of 

rci-ir i to rulcB is said to be an "extensible function". The "ALSO" 

clause is used to add cases to an extensible function: 

RULES OF ULISP ALSO = 

IF <riLISP>:X THEN <nLISP>!Y ELSE 

-^ (niSSING EXPRESSION AFTER ELSE) ®ERROR, 

IF <nLISP>:X THEN 

-> (niSSING EXPRESSION AFTER THEN) ©ERROR ; 

Extensions can be made effective throughout the program or only 
in the current block, as the user wishes. 

A regular LAMBDA function can also be extended. Its bound 
variables are considered analogous to a DEC and its body analogous to 
a REG. Accordingly, the compiler converts it to an equivalent 
reiirite function of one rule before extending it. 
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THE EXTENSIBLE COflPILER 

To make an extensible compiler practical, the casual user must 

bo able to understand how it works in order to change it. To 

demonstrate that it is not inordinately difficult to understand the 

L.ISF70 compiler, those rules which get involved in translating a 

particular statement from HLISP to LAP/PDP-10 are shown below. A 

Bim|:il i f ied LISP70 (type I ess and unhierarchical ) is used in the 

Gxamplos, but th(3 real thing is not much more complicated. The 

slatement to be translated is: 

IF A < B THEN C ELSE D 

The rules invoked in the IILISP-to-LISP translator are: 

RULES OF riLISP = 

IF <nLISP>:X THEN <I1LISP>:Y ELSE <nLISP>!Z 
-* (COND (:X :Y) (T :Z)), 

:X '< :Y ^ (LESSP :X :Y), 

:VAR -> :VAR ; 

Tl-)e l.rSP-to-i1L compiler below utilizes the following feature: if 
c\ colon variable occurs in the REC but it did not occur in the DEC, 
an "existential value" (which is something like a generated symbol) 
Ig bound to it. Here, the existential value is used as a compiler- 
generated I al:)e I . . 
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RULES OF COnPILER = 



(COND (T :E)) 

-> :E @COnPILER, 

(COND (:B :E) ...) 

-. :B oCOMPILER 

(BRANCH_FALSE :ELSE) 
:E @COnPILER 
(BRANCH :OUT) 
(LABEL :ELSE} 
(COND ...) ©COnPlLER 
(LABEL :OUT), 

(LESSP :A :B) 

-. :A ©COdPILER 
(PUSH_DOWN) 
:B ©COMPILER 
(COMPARE LESS) , 

:V -. (LOAD :V) ; 



The ML"to-LAP translator below assumes that the value register 
of the ideal machine is represented on the PDP-10 by a register named 
"VAL", that there is a single stack based on register "P", and that 
variables can be accessed from fixed locations in memory. 
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RULES OF ML = 



{BRANCH_FALSE :LBL) 

-> (JUnPE VAL :LBL), 

(BRANCH :LBL) 

-. (JRST jLBL), 

(LABEL :LBL) 
•^ :LBL, 

(PUSH_DOWN) 

-. (PUSH P VAL) , 

(COnPARE LESS) 

-^ (CAMGE VAL P) 
(TDZA VAL VAL) 
(HOVE I VAL 1) 
(POP P), 

(LOAD :V) 

-. (HOVE VAL :V) ; 



"he code generated is thus; 

(HOVE VAL A) 
(PUSH P VAL) 
(HOVE VAL B) 
(CAHGE VAL P) 
(TDZA VAL VAL) 
(HOVE I VAL 1) 
(POP P) 

(JUHPE VAL E0001) 
(HOVE VAL C) 
(JRST E0002) 
E0001 (HOVE VAL D) 



E0002 
Peephole optimization guided by another rewrite function can reduce 

this to six instructions. 
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AUTOriATIC ORDERING OF REWRITE RULES 

In most pattern matchers, candidate patterns to match an input 
stream are tried either in order of appearance on a list or in an 
oGGontially random order not obvious to the programmer. LISP70 tries 
inatcl-ies in an order specified by an "ordering function" associated 
i.iitt"! eacl-i set of rewrite rules. 

One common ordering is "BY APPEARANCE", which is appropriate 
ij|-)(M-) thie programmer wants conscious control of the ordering. Another 
is "BY SPECIFICITY", which is useful in I eft- to-right parsers and 
otlier applications where the compiler can be trusted to order the 
ruloG GO that more spocific cases are tried before more general ones. 
Nhen neither of these standard functions is appropriate, the 
[u-ogrammor can define and use specialized ordering functions, or can 
extend SPECIFICITY to meet the special requirements. 

Automatic ordering is a great convenience for a user who is 
extending a compiler, a natural language parser, or an inference 
sgstem. It can eliminate the need to study the existing rules simply 
to determine where to position a new rule. Ordering functions can 
also l:)e designed to detect inconsistencies and ambiguities and to 
discover opportunities for generalization of' similar rules. 
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As an exatiifole, take the LISP-TO-ML translator "COIIPILER", which 
includes the following rule for the intrinsic function PLUS (slightly 

G i inp I i f i ed for presentation): 

RULES OF COMPILER = 

(PLUS :X :Y) 

-^ :X@COnPILER 
(PUSH_DOUN) 
:Y@C0f1PILER 
(ARITHI1ETIC ADD) ; 

To add special cases to the compiler for'sums including the 
constant zero, the user could include the following declaration in a 
|;irogram: 

RULES OF COMPILER ALSO = 

(PLUS :X 0) ^ :X@COnPILER, 
(PLUS e :X) -* :X@COnPILER ; 

The compiler Is ordered by SPECIFICITY, which knows that the 
literal is more specific than the variable :X or :Y. Therefore, 
l.)oth of the new rules would be ordered before the original PLUS rule. 
Sui)t:)ose the added rules were placed after the general rule; then the 
oi-icjinal rule would get first crack at every input stream, and sums 
with zero would not be processed as special cases. 
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AN ORDERING FUNCTION 

Tl'ie c:oiiij)lete definition of tlie ordering function SPECIFICITY is 
iDoyond the Gcopo of this paper. It works rouglily as follows. 
Cornt:)aring DEC patterns by a left-to-right scan, it considers literals 
more specific than variables, a colon variable at its second 
occurrcMice more specific than one at its first occurrence, and a 
function call with an "@" more specific than a variable but less 
specific than a literal. The specificity of a replacement <F> is 
that of the most general rule in the function F. 

A DEC with an ellipsis is considered to expand to multiple rules 

in which the ellipsis is replaced by 8, 1, 2, 3, ... w consecutive 

variables. The specificity of each expanded rule is considered 

separately. Observe that between two expansions of an elliptic rule 

some other rewrite rule of intermediate specificity may lie.' 

Example: 

RULES OF SILLY = 

A ... B ... C ^ 1, 
A B :X :Y -^ 2 ; 

Two of the expansions of the first rule are: 

A B :X C ^ 1, 
A :Z B C -^ 1, 

and the second rule of SILLY comes between these in specificity. 
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SF^ECIFICITY is itself defined by a system of rewrite rules. To 

fiivr: a flavor- of lioii this is done, a very simplified SPECIFICITY ui M 

lie defined. It takes two arguments (DEC patterns translated to LISP 

n o i: n t i o n ) a n d r e t u i- n s them in the proper order. 

RULES OF SPECIFICITY = 

(COLON :V) (LITERAL :L) 

-^ (ORDER (LITEF^AL :L) (COLON :V)), 

(LITERAL :L) (COLON :V) 

-. (ORDER (LITERAL :L) (COLON :V)) ; 
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OTHER FACILITIES AND APPLICATIONS 

Other facilities of the rewrite system include side-conditions, 
noniunctive match, disjunctive match, non-match, repetition, 
evaluation of LISP and NLISP expressions, look-ahead, look-behind, 
and revei'-sible rules. 

Out of rcu-jrlte functions, it is easy to define systems of 
inference rules, assertions, and beliefs. LISP70 has facilities for 
retrieving either all or the first of the assertions in a set of 

r u I e s 1 1-) a t m a t c I'l a given pattern. 

Reurite i-ules are a great help in natural language 
understanding, uhether the methods used are based on phrase structure 
rirrtmiiiar, feat4;*-^es, keywords, or word patterns. A use of LISP70 with 
the latter iiiet|-)Od is described in a companion paper [9]. 



19 Tesler, Enea, and Smith 

CONCLUSIONS 

Nany of the design decisions of LISP70 are contrary to trends 
seen in other "successors to LISP". The goals of these languages are 
similar, but their means are often quite diverse. 

Concern uith good notation does not have to compromise the 
dGvelopment of powerful facilities; indeed, good notation can make 
those facilities more convenient to use. 

Emphasis on pattern-directed computation does not overly 
constrain the programmer accustomed to writing algorithms. Rewrites 
and algorithms can be mixed, and the most appropriate means of 
defining a given function can be selected, 

LISP70 does not limit the use of pattern rewrite rules to a few 
faciiities like goa I -achievement and assertion-retrieval. A set of 
rules can be applied to arguments like any other function, and can 
stream data from any type of structure or process to any other. 

■'.Automatic ordering does not prevent the programmer from seizing 
control, but allows him to rel inqui sh control to a procedure of his 
osing to .save him tedious study of an existing program when making 

xtens i ons. 



c h o 
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Tho LISP7B karnel is being debugged on a PDP-10 at the time of 
thlL; i-iriting (February, 1973). The language has already been used 
iajccessful ly in programs for question-answering and planning. After 
the kernel can reliably compile itself, extensions are planned to 
improve its control structure, editing, and debugging capabilities, 
nnci versions may be bootstrapped to other computers. 
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